Recognizing patterns within a set of data is important in many fields, including speech recognition, image processing, seismic data, etc. Some image processors collect image data and then pre-process the data to prepare it to be correlated to reference data. Other systems, like speech recognition, are real time where the input data is compared in real time to reference data to recognize patterns. Once the patterns are “recognized” or matched to a reference, the system may output the reference. For example, a speech recognition system may output equivalent text to the processed speech patterns. Other systems, like biological systems may use similar techniques to determine sequences in molecular strings like DNA. If the real time data processing is too intensive for one processing unit (PU), then parallel PUs may be employed to increase computational power. Most of the processing tasks are numerically intensive as matrix and statistical techniques are used to determine a “closest” match between input data and reference data. It may be rare for the comparisons to be exact matches. While many fields use pattern recognition with similar techniques, image processing is discussed in the following simplified explanations.
Image processing and analysis may be defined as the “act of examining images for the purpose of identifying objects and judging their significance.” Image analysts may study remotely sensed data and attempt, through logical processes of detecting, identifying, classifying, and measuring, to evaluate the significance of physical objects, their patterns and spatial relationship. The analyzed image data usually is converted to a digital form for analysis.
In a most generalized way, a digital pattern may be an array of numbers depicting a distribution of certain field parameters (such as reflectivity of electro-magnetic radiation, emissivity, temperature or some geophysical or topographical elevation. For example, a digital image comprises discrete picture elements called pixels. Associated with each pixel is a number (digital number, DN), that depicts the average radiance of relatively small area within a scene. The range of DN values being normally 0 to 255 in this case corresponding to 8 binary bits. The size of pixel affects the reproduction of details within the scene or image. As the pixel size is reduced, more scene detail is preserved in digital representation.
Remotely sensed data may be recorded in digital forms and then processed by computers to produce data for interpretation purposes. For example, images are usually available in two forms, photographic film or in the form of a set of digital data. Variations in image characteristics are represented as variations in brightness on photographic films or in variations in the data representing brightness. A particular part of an image reflecting more energy will appear bright while a different part of the same image reflecting less energy will appear black. These variations contain information that may be interpreted by processing.
Classification of sensed data may be used to assign corresponding levels with respect to groups with homogeneous characteristics, with the aim of discriminating multiple objects from each other within an image or other data. Classification may be used in formatting reference data and in preprocessing input data during the pattern recognition process. In the case of an image, classification may be executed on the base of spectral or spectrally defined features, such as density, texture, etc. in the feature space. It may be said that classification divides the feature space into several classes based on a decision rule.
In many cases, classification is undertaken using a computer, with the use of mathematical classification techniques. Classification may be made according to procedures, which define selections of features that allow discrimination between classes. Features used to discriminate between the classes may be established using multi-spectral and/or multi-temporal characteristics, textures, etc. Training data may be sampled in order to determine appropriate decision rules. Classification techniques such as supervised or unsupervised learning may then be selected on the basis of the training data sets. Various classification techniques are compared with the training data, so that an appropriate decision rule is selected for subsequent classification.
In image processing, depending up on the decision rule, all pixels are classified in a single class. There are two methods of pixel-by-pixel classification and per-field classification with respect to segmented areas. Popular techniques include multi-level slice classifier, minimum distance classifier, and maximum likelihood classifier. Other classifiers such as fuzzy set theory and expert systems may also be used.
Clustering is a method of grouping data with similar characteristics. Clustering may be divided into hierarchical clustering and non-hierarchical clustering. In hierarchical clustering, the similarity of a cluster is evaluated using a “distance” measure. The minimum distance between clusters will give a merged cluster after repeated procedures from a starting point of pixel-wise clusters to a final limited number of clusters. The distances to evaluate the similarity may be selected using the following methods:                Nearest neighbor method wherein the nearest neighbor with the minimum distance is used to form a new merged cluster.        Furthest neighbor method wherein the furthest neighbor with a maximum distance is used to form a new merged cluster.        
Centroid method wherein the distance between the gravity centers of two clusters is evaluated for merging a new merged cluster.
Group average method wherein the root mean square distance between all pairs of data within two different clusters, is used for clustering.
Ward (root mean square) method wherein the root mean square distance between the gravity center and each member is minimized.
A minimum distance classifier is used to classify unknown data into classes that minimize the distance between the data and the class in multi-feature space. The distance is defined as an index of similarity so that the minimum distance is identical to the maximum similarity. The distances often used in this procedure of distance classification include the Euclidean distance and the Mahalanobis distance. The Euclidean distance is used in cases where the variances of the population classes are different to each other. The Euclidean distance is theoretically identical to the similarity index. A normalized Euclidean distance is proportional to the similarity index. The Mahalanobis distance is used where there is correlation between the axes in feature space.
Closest match determination is used in many applications like image processing or in image classification and is a very computationally expensive task. Hardware is needed for real-time applications but existing hardware solutions have some major limitations concerning scalability. If more integrated circuits (ICs) are used in order to increase the number of reference patterns (RPs) processed, external circuits and buses are needed. Likewise, if a number of RPs are reloaded, some extra computations may be also needed.
Typically, an Application builds a list of input patterns (IPs) using the techniques discussed. The Application then sends all the IPs to a minimum distance classifier that calculates distances (e.g., Euclidean distances) according to techniques discussed. The role of this minimum distance classifier is to process each of the IPs and to compute the distance between a particular IP and all the reference patterns (RPs). Comparison circuits are used to find a minimum distance that corresponds to one of the RPs. Each of the RPs has a specific, unique identification (ID). When the minimum distance is determined, the minimum distance classifier can output the minimum distance and the ID for the corresponding RP. For real-time applications, an important metric is the performance bandwidth (the number of IPs that can be processed per second relative to the number of desired RPs).
A common way to find a minimum distance within a reasonable time is to use a priority process. This is commonly done by scanning all the bits of the distances beginning at the most significant bit (MSB) and ending at the least significant bit (LSB). In order to do this, all the RPs have to be first loaded in each processing unit (PU) which then sequentially computes the distance to each corresponding reference pattern for each input pattern. Next, the minimum distance across all the reference patterns is determined.
There are several problems with this prior art method. To get a minimum distance, a common output bus is needed to couple all the distances to a comparison circuit to determine the minimum one. To achieve a reasonable speed, there is also a need to use a priority scheme to obtain the minimum distance. To implement this priority scheme, a common bus and merge circuitry are also needed. The merge circuitry and the bus use a great deal of area on an IC chip used to implement this function. The common bus also makes the physical scalability more complex. When more processing units (PUs) are needed than can be integrated on a single IC, a common bus must be implemented which extends outside of each IC chip. Also, to get the minimum distance with a priority scheme, several clock cycles are needed. Because one also needs common circuitry to merge together all PUs, the process is relatively slow. If one has to merge several ICs together, the corresponding circuitry may have a low clock rate because of all timing constraints of off-chip communication. Because of transmission line effects, off-chip buses typically run an order of magnitude slower than on-chip buses. Therefore, the prior art implementation of finding the minimum value is typically slower than desired. Another problem occurs if the number of RPs is greater than the number of available PUs. In this case, all minimum distances for a first set of RPs must first be determined and then the PU must be reloaded with a second set of RPs to compute all minimum distances again. Both results must be then merged together by selecting the minimum distance for a certain pattern. This must be repeated for each set of patterns and is relatively slow, causing major scalability problems with respect to the number RPs.
There is, therefore, a need for a method and an apparatus to allow improved scalability and fast closest match when processing patterns in pattern recognition systems. It is further desirable to be able to expand the number of PUs or the number of RPs without requiring major increases in circuitry or reductions in processing speed.